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PREFACE 

Mid-continent Research for Education and Learning (M cREL) , located in Aurora, 
Colorado, is a private, nonprofit organization founded in 1966. McREL’s mission 
is to make a difference in the quality of education through applied research, product 
development, and service. 



T his publication was created through 
McREL’s contract with the U.S. 
Department of Education’s Institute 
of Education Sciences to serve as 
the regional educational laboratory for 
the Central Region states of Colorado, 
Kansas, Missouri, Nebraska, North Dakota, 
South Dakota, and Wyoming. In this role, 
McREL is implementing a comprehensive, 
problem-based research, development, and 
service program designed to create the 
knowledge, tools, and strategies needed 
to transform low-performing schools into 
high-performing learning communities. 

For more than a decade, McREL has been at 
the forefront of research, practice, product 
development, and evaluation related to 
standards-based education. McREL’s na- 
tional leadership area under the regional 
laboratory contract is standards-based edu- 
cational practice. 

This issue of Noteworthy, written primarily 
for school and district leaders, policymak- 
ers, and program administrators, draws 
largely from McREL’s recent research syn- 
thesis, The Effectiveness of Out-ofSchool - 
Time Strategies in Assisting Low-Achieving 
Students in Reading and Mathematics (Lauer, 
Akiba, Wilkerson, Apthorp, Snow, & 
Martin-Glenn, 2004). This publication 
represents part of McREL’s continu- 



ing efforts to build on its expertise, turn 
research into practical guidance, and 
work with schools, districts, and states to 
improve their practices and capitalize on 
the great potential that standards-based 
education holds for students. 

There are a number of out-of-school-time 
research reports that are not presented 
here because program effects on at-risk 
students were not studied, or because they 
did not meet the rigorous design criteria re- 
quired for inclusion in McREL’s synthesis. 
These other studies are, nonetheless, good 
sources of information and programmatic 
ideas, and they are worthy of review. This 
Noteworthy, however, is intended to bring 
a research-based perspective to those who 
seek to enhance their understanding of 
out-of-school-time programming for at-risk 
students. The authors sincerely hope that 
readers will find this publication to be an 
informative and practical resource for use 
in understanding, evaluating, and design- 
ing out-of-school-time programs. 

The authors wish to acknowledge the con- 
tributions of a number of individuals in 
the preparation of this publication. In par- 
ticular, thanks go to the other authors of 
McREL’s recent synthesis, Patricia Lauer, 
Helen Apthorp, Stephanie Wilkerson, 
Mya Martin-Glenn, and Motoko Akiba. 
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Without their work, along with the assis- 
tance of Zoe Barley, Becky Van Buhler, Terry 
Young, Barbara Aiduk, and Robyn Alsop, 
this publication could not have been written. 
Appreciation also is extended to external re- 
viewers Joyce Bales and David Waters, and 
to McREL staff members Elena Bodrova, Lou 
Cicchinelli, Maggie Cooper, Bryan Goodwin, 
Jodi Holzman, Danette Parsley, and Adrienne 
Schure for their helpful comments. The 
authors also would like to acknowledge Brian 
Lancaster’s design of the cover of this publi- 
cation and desktop publishing assistance. A 
debt of gratitude also is owed to the authors 



of the research that supports this publication. 
Last, but certainly not least, special thanks to 
Barbara Gaddy and Ceri Dean for their work 
on this project. With vision and care these 
two educators have brought this Noteworthy 
though many iterations to this final version. 
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chapter ■ Introduction 



A t some point in their lives, ev- 
eryone has said it: there aren’t 
enough hours in the day. For 
many children with learning 
difficulties, and their teachers, this old 
maxim rings all too true. When children 
fall behind in a content area or areas, there 
may not be sufficient time in the school 
day to offer the remedial instruction neces- 
sary to get them up to speed. One potential 
solution for making up this deficit is out-of- 
school time, or OST, programs. 

As the name implies, out-of-school time 
programs are targeted to the hours that 
school-age children are not in school. 
The most common OST formats are 
after-school and summer-school programs. 
Researchers also have studied before- 
school and Saturday school programs. 

In 2001, 6 million of the 54 million K-8 
children in the United States participated 
in after-school programs (De Kan ter, 
2001). These programs were either com- 
munity-sponsored after-school programs 
or school-based extended-day programs. 
Between 1994 and 2001, the number of 
schools offering after-school programs 
doubled (De Kanter, 2001). But according 
to the National Institute on Out-of-School 
Time (2004), millions of children between 
the ages of 6 and 12 do not participate in 
any kind of adult-supervised after-school 
activity. Advocates of after-school pro- 



I I IM THIS CHAPTER 

• The Evolution of OST 
programs 

• Past Research 

• The McREL Out-of-School Time 
Research Review 

grams have noted that there is increasing 
public support for the development and 
funding of after-school programs in public 
schools (The After-School Corporation, 
1999; Fashola, 2002). 

This chapter provides historical back- 
ground on OST programs in the United 
States, a look at the context within which 
OST programs are now being developed, 
and an overview of the body of available 
OST research. The chapter also briefly 
describes the McREL synthesis framework, 
on which this issue of Noteworthy is par- 
tially based. 

The Evolution of OST Programs 

Though out-of-school time programs as we 
now know them (e.g., after-school, summer 
school) were not fully implemented in the 
United States until the 1970s and 1980s, 
the roots of OST run far deeper, extend- 
ing as far back as the late 1800s (Harvard 
Family Research Project, 2000). Out-of- 
school time programs first began to receive 
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federal funding during World War II, 
when the U.S. government began funding 
after-school programs as a means of pro- 
viding childcare for women entering the 
workforce for the first time. Government 
funding of these programs ceased with the 
end of the war, when many women again 
left the workforce. 

In the 1950s, formal summer school pro- 
grams emerged, originally as a potential 
solution for the prevention of behavior 
problems. Summer school administra- 
tors, however, quickly began to see these 
program sessions as a good avenue for re- 
mediation. 

A number of societal factors have been 
cited by researchers as affecting the push to 
create summer school programs reflecting 
an academic emphasis, including (1) family 
influences (e.g., maternal employment and 
single parent households), (2) the need for 
the United States to maintain a globally 
competitive education system, and (3) the 
emphasis on rigorous learning standards 
and minimum student proficiency re- 
quirements. Cooper, Charlton, Valentine, 
and Muhlenbruck (2000) contend that, 
although additional purposes of summer 
school programs will likely continue to be 
identified, their focus on academic reme- 
diation will probably always remain. 

Historically, meeting the needs of low- 
income children has been a primary reason 
for the development of OST programs, 
particularly after-school programs. Because 
safety tends to be a greater concern in 
low -income neighborhoods than it is in 



middle-income neighborhoods, there is a 
greater need for low-income children’s out- 
of-school time to be supervised by adults. 
However, children from low-income fami- 
lies are less likely to have after-school care- 
givers available in their homes. 

The push to “even the odds” for the 
nation’s low- income children began to 
pick up momentum in the 1960s. During 
his 1964 Presidential campaign, Lyndon 
Johnson spoke of an America “where no 
child will go unfed and no youngster will 
go unschooled.” Title I of the original 



FROM THE FIELD: 

What advantages do you see in serving 
at-risk students in OST programs? 

In our after-school classes, children are 
often in a class of their interest and/or a 
subject they need extra help with. These 
classes are smaller in size than a regular 
classroom, so the students receive more 
one-on-one attention. With the smaller 
teacher ratios, children can get the 
individual help and attention they need, 
thus increasing their skill level, which 
in turn leads to higher self confidence. 
Children also get the opportunity to meet 
other children their age and/or work in a 
multi-age setting. 

Denise Southwell, Teacher 
Saline Community Education 
Saline, Ml 
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Elementary and Secondary Education Act 
was created in part because of data indicat- 
ing that low-income children are at risk for 
academic failure. It became evident that 
low-income children were in need of ad- 
ditional educational time to supplement 
what they experienced during 
regular school hours (Cooper et 
ab, 2000; Borman & D’Agostino, 

1996). Despite concerns that 
after-school programs that em- 
phasized academics would in- 
terfere with time spent in play, 
these programs have become 
the norm as educators search 
for ways to meet the needs of 
their low-income or otherwise 
at-risk students (Halpern, 2002). 

Concerns about the social needs 
of children have, in many cases, 
led to the incorporation of social 
components into OST programs. 

Recent developments in education leg- 
islation will likely intensify the discus- 
sion about the potential benefit of OST 
programs to increase the achievement of 
at-risk students. Under the No Child Left 
Behind Act, all schools and states are held 
accountable for ensuring that all students 
reach or exceed each state’s proficient level 
of performance in reading and mathematics 
by the end of the 2013-2014 school year. 
States are required to report disaggregated 
scores for a number of student subgroups, 
including economically disadvantaged stu- 
dents, students with disabilities, students 
from major racial and ethnic groups, and 
students with limited English proficiency. 
Disaggregating scores in this manner pro- 



vides a clearer picture of the progress of 
different groups of students — informa- 
tion educators can use to target and refine 
educational programs, including programs 
outside the traditional school day. 



Past Research 

A wealth of research on OST programs 
has been conducted over the years. These 
studies are as varied as the programs them- 
selves. Most of the studies have focused 
on a specific program or program type, 
but all have shared the goal of distilling 
information about what makes a particu- 
lar program effective. Because of the large 
number of studies of OST programs, this 
section is limited to descriptions of reviews 
and syntheses of OST research. 

The first of these is a comprehensive 
synthesis of summer school research 
conducted by Cooper and his colleagues 
(Cooper et ab, 2000). Cooper et al. 
reviewed 93 program studies to better 
understand the effects of summer school 
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on student achievement. Their results in- 
dicated that summer school had positive 
academic effects on both middle-income 
and low-income students. Positive effects 
on student performance also were found 
for programs for smaller numbers of stu- 
dents and for programs that provided stu- 
dents with individualized and small-group 
instruction. Cooper et al. also found that 
students in the early elementary and sec- 
ondary grades benefited more from summer 
school classes compared to students in late 
elementary grades. 

McComb and Scott-Little (2003) provide 
a narrative review of 27 studies of after- 
school programs. By comparing study 
results, they concluded that large varia- 
tions in program content, size, goals, and 
research designs prevented a simple answer 
to the question about the effects of after- 
school programs on academic outcomes. 
Instead, they focused their discussion on 
the conditions that favored positive student 



more from OST programs than did high- 
achieving students, and that students who 
attended the programs more frequently 
benefited more. Overall, however, the 
authors were not able to reach any strong 
conclusions. 

The lack of research to support conclusions 
also was reported in an earlier after-school 
synthesis conducted by Fashola (1998), 
who reviewed evaluations of 34 programs 
delivered in extended-day or after-school 
formats. The author concluded that the re- 
search reveals many promising approaches 
but that it is limited in the information it 
can provide. 

Miller (2003) reported on a comprehensive 
narrative review of after-school programs 
for middle school children. The purpose 
of Miller’s report was to examine the roles 
of after-school programs in promoting aca- 
demic success and positive early adolescent 
development. Miller described the effects 
of different after-school programs 
on academic outcomes and on 
outcomes that Miller and others 
connect with academic success, 
such as students’ attitudes toward 
school. Although the report 
provided valuable information 
related to all facets of how after- 
school programs can benefit ado- 
lescent development, questions 
about specific effects on achieve- 
ment in reading and mathematics 
were left unanswered. 




outcomes. For example, there were indica- One study of OST programs that received 
tions that low-achieving students benefited national attention is the first-year evalu- 
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ation of the 21st Century Community 
Learning Centers program (U.S. Depart- 
ment of Education, 2003). Congress au- 
thorized this program in 1994 to promote 
broader use of schools by communities and, 
in 1998, redesigned 
the program to provide 
academic as well as 
recreational activities 
to students outside of 
regular school hours. 

The first-year evalu- 
ation compared the 
academic and devel- 
opmental outcomes 
of elementary and 
middle school stu- 
dents who attended a 
21st Century program 
with those who did 
not attend. 

In general, first-year findings were discour- 
aging. No significant impacts on achieve- 
ment were found in reading or math- 
ematics for elementary or middle school 
students. As a result, it is not possible to 
link a specific 21st Century program to 
outcomes of the students served by that 
program. As the authors (U.S. Department 
of Education, 2003) noted, “The study was 
designed to examine the characteristics 
and outcomes of typical programs and 
did not attempt to define the character- 
istics of the best programs” (p. xi). In a 
footnote they added, “This study focuses 
on school-based programs that are part of 
the 21st Century program. Results do not 
extrapolate to all after-school programs in 
general” (p. xi). In other words, the evalu- 



ation sought to address the effectiveness 
of the 21st Century grant program as a 
funding source and not the effectiveness of 
after-school strategies in general. 

Proposed funding cuts 
to the program as a 
result of the first-year 
evaluation drew criti- 
cism from some re- 
searchers and evalua- 
tors of OST programs, 
who contended that 
one year of findings 
was an insufficient 
basis on which to 
pass judgment about 
program effective- 
ness (Harvard Family 
Research Project, 
2003). A number of 
critics have called 
for consolidating knowledge gleaned from 
many individual evaluations to better ap- 
proximate the effects of after-school inter- 
ventions, the approach used for McREL’s 
synthesis. 

Second-year findings for the 21st Century 
grant program were released in October 
2004 (Dynarski et ak, 2004). Some posi- 
tive effects were reported. For example, 
middle school social studies achievement 
was higher, and elementary school students 
reported higher feelings of safety. Overall 
findings were consistent with the first-year 
evaluation, however. 

Reactions to the second-year findings 
have not yet been widespread, making the 
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impact of this evaluation unclear. Also, it 
is important to note that despite the con- 
troversy over the initial evaluation, these 
studies are making an important contribu- 
tion to our knowledge about OST programs 
by documenting the great variation in the 
characteristics of programs across school 
districts, particularly in the range of activi- 
ties offered and in the relative emphasis on 
academic assistance. 



After reviewing thousands of 
studies, McREL researchers iden- 
tified 27 reading studies and 33 
mathematics studies that met the 
rigorous design criteria required 
for inclusion in the synthesis. 
Each of these studies compares 
an OST treatment of at-risk 
public school students to the 
performance of program non-par- 
ticipants. McREL’s synthesis contributes to 
the knowledge base about OST strategies 
for low-achieving or at-risk students in the 
following ways: 



• By examining research on OST strat- 
egies delivered in all timeframes, 
including summer school, after school, 
extended day, before school, vacation 
sessions, and Saturday schools. 



progress. Because instruction for 
supplementary services must occur 
outside the regular school day, 
there is interest among educa- 
tors in the effectiveness of OST 
strategies for improving student 
achievement. 



The McREL Out-of-School 
Time Research Review 

McREL recently completed a review of 
research on OST programs for at-risk stu- 
dents (Lauer et ak, 2004). The impetus for 
this work was the “supplemental services” 
provision of the No Child Left Behind 
Act (NCLB), which requires that states 
offer supplementary education services 
to low- income students in Title I schools 
that do not achieve adequate yearly 



• By including the results of separate 
analyses of the effectiveness of OST 
strategies for student achievement in 
reading and in mathematics. 

• By including studies only if the design 
involved a comparison group of stu- 
dents who did not experience the OST 
strategy under investigation. 

• Studies were coded for alignment 
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with criteria of research quality, and 
synthesis results were described in re- 
lationship to these ratings. 

An important point to keep in mind is that 
McREL’s review of the research focused on 
whether or not OST programs provided 
benefits specifically for at-risk students. To 
qualify for inclusion in the synthesis, the 
study had to examine the effectiveness of 
an OST strategy for low-achieving stu- 
dents or students at risk for school failure. 
The study could include students perform- 
ing at other achievement levels, but it had 
to disaggregate effects for those entering 
an OST program with low achievement or 
at risk for low achievement. McREL’s goal 
was to assess the effectiveness of OST strat- 
egies for those students who are most likely 
to need them. Low achievement could 
be determined by student performance 
on standardized tests or classroom assess- 
ments or through teacher-assigned grades 
or recommendation for assistance. At-risk 
status could be determined by characteris- 
tics typically associated with lower student 
achievement and school dropout in large- 
scale data collections including low socio- 
economic status (SES), racial or ethnic mi- 
nority background, a single parent family, a 
mother with a low level of education, and 
limited proficiency in English (Slavin & 
Madden, 1989; Miller, 1993). 

McREL’s analysis of the available re- 
search pointed to small overall positive 
effects in both reading and mathematics 
programs. Within the synthesis, impacts 
were reported as effect sizes. For this 
document, however, the effect sizes have 



been converted into percentile point 
gains. The synthesis authors identified 
an average 5 percentile point gain for 
reading and a 6.5 percentile point gain for 
mathematics programs. Put another way, a 
student scoring at the 50th percentile in 
reading, who attained average gains from 
an OST program, would increase his or 
her achievement to the 55th percentile. 
For students scoring close to proficiency 
cut scores, such a gain could be quite 
significant. 

More important, McREL’s review of the 
research identified specific program char- 
acteristics and approaches associated with 
improvements in reading and in math- 
ematics. The following chapters include 
examples that illustrate these characteris- 
tics and approaches. This information can 
be used by OST program leaders to inform 
program design and implementation. 
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Effective OST 
Reading Programs 



P roficient readers often take 
the act of reading for granted. 
Whether we’re glancing at street 
signs, reading the morning paper, 
or reviewing e-mail, reading is just one 
more thing we do as we go about our daily 
routines. For many children, however, 
learning to read is far from simple. 
Results from the National Assessment of 
Educational Progress (NAEP) indicate 
that a large percentage of students are not 
meeting reading standards. In 2003, just 31 
percent of fourth-graders, and 32 percent 
of eighth-graders, performed at or above 
the proficient level 1 in reading (National 
Center for Education Statistics, 2004). 
Among children who are eligible for free 
or reduced-price lunch, a key indicator of 
poverty, just 15 percent of fourth graders 
and 16 percent of eighth graders performed 
at the proficient level. 

Within the current framework of start- 
dards-based reform and accountability, all 
schools and districts are required to ensure 
that every child can read and understand 
both literary and informational texts by the 
end of third grade. For any child, this is no 
small task. To achieve reading proficiency, 
students must master certain knowledge 
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and skills at or before critical grade levels. 
During the primary years (K-2), children 
need to master the reading fundamentals, 
such as associating sounds with written 
words. During the intermediate grades 
(3-5), children need to develop and use 
all word identification concepts and skills, 
as well as comprehension strategies such 
as recognizing confusion, adjusting one’s 
strategies, and identifying and summarizing 
main ideas and important details (McREL, 
n.d.). As children prepare for and progress 
through middle school and high school, 
they are expected to develop and use ad- 
vanced reasoning for reading so that they 
can understand and interpret texts well 
enough to take and pass a college-prepara- 
tion sequence of courses (Committee for 
Economic Development, 2000). 



'Three achievement levels have been defined by NAEP (2004): 

Basic — Partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade. 
Proficient — Solid academic performance for each grade assessed. Students reaching this level have demonstrated 
competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to 
real-world situations, and analytical skills appropriate to the subject matter. 

Advanced — Superior performance, (see http://nces.ed.gov/nationsreportcard/reading/achieve.asp) 
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When children don’t master these increas- 
ingly complex reading skills on schedule, 
the negative effects spill over to other 
content areas. Struggling readers tend 
to fall farther and farther behind other 
students, not only in language arts, but 
in other subjects as well. Research shows, 
however, that this trend can be turned 
around. According to Slavin, Karweit, and 
Madden (1989), “the negative spiral that 
begins with poor achievement in the early 
grades can be reversed” (p. 4). 




To help students attain proficiency in 
reading, many educators are considering 
out-of-school time (OST) strategies and 
programs. These educators are looking 
for effective programs to mitigate summer 



learning loss, remediate skill deficiencies, 
accelerate learning, and prepare students 
for the intellectual challenges of later 
schooling and work. In addition to address- 
ing these academic focuses, OST strategies 
and programs enable educators to address 
the safety, behavioral, cultural, vocational, 
emotional, and social needs of students. 

The timeframes for delivering OST strate- 
gies include after school, Saturday school, 
and summer school. The variation among 
the purposes and formats of these strategies 
reflects how interventions address the dif- 
ferent academic and social learning needs 
of students. The National Institute on 
Out-of-School Time “believes that high- 
quality after-school programs focus on the 
development of the whole child, integrat- 
ing academic supports such as literacy skills 
into programming that also promotes chil- 
dren’s social, emotional, and physical de- 
velopment” (Hynes, O’Connor & Chung, 
1999, p. 1). Others have emphasized the 
informality of after-school programs as 
being well suited to developing the social 
and cultural dimensions of literacy, such 
as helping children see how reading and 
writing can be intrinsically rewarding 
and relevant to their lives (Spielberger & 
Halpern, 2002). 

Program developers seeking to design or 
strengthen OST interventions for their 
struggling students can find some useful 
guidance from research on the effective- 
ness of OST programs and strategies. Find- 
ings from McREL’s analysis of research, for 
example, point to potentially effective 
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ways of providing students with instruction 
and related experiences that can help them 
advance their reading achievement. 

Out-of-School Time Research 

McREL, in its review of the research on 
OST strategies in reading, evaluated five 
program characteristics to determine their 
influence on the overall effectiveness of 
OST programs: (1) timeframe (summer 
school, after school, Saturday school), 
(2) grade level, (3) activity focus, (4) 
program duration, and (5) student group- 
ing. On average, OST strategies improved 
the reading achievement of participating 
low-achieving or at-risk students by five 
percentile points. Although the avail- 
able research is not definitive, McREL’s 
analysis, as well as guidance from other 
studies, offers a number of points that OST 
program developers may want to consider. 

Targeting Interventions 

Research has long suggested that programs 
and strategies focused on the prevention of 
reading difficulties in elementary students 
are most effective when they are delivered 
to children early in their schooling, before 
reading problems become entrenched 
and self-esteem issues stall the learning 
process (Mathes, 2003). The same appears 
to hold true of OST strategies for reading. 
Programs that target students in grades 
K-2 show average gains of 9 percentile 
points (Lauer et ah, 2004). Conversely, the 
gains in achievement for middle school 
students were minimal, and students in 
third through fifth grades actually showed 
declines in achievement. The research 
also indicates, however, that OST strate- 



Short-Term Summer 
School Intervention 



Location: 

• Ohio 

Students served: 

• First- through fourth-grade 
at-risk students 

Program components: 

• Small-group tutoring 

• Phonics instruction 

• Computer-assisted instruction 

Program outcomes: 

• More than six-month grade- 
equivalent gain in reading 
proficiency for first graders 

gies also can have a significant positive 
impact for students in upper grade levels. 
High school students seem to reap strong 
benefits from OST programs; Lauer et 
al. (2004), for example, found that these 
students posted average gains of more than 
eight percentile points. These findings 
corroborate earlier studies suggesting that 
younger and older students benefit most 
from OST programs in reading. 

Luftig’s (2003) report on a 2-week summer 
reading program illustrates some of the 
benefits of early intervention. As part of 
the program, at-risk students entering first 
through fourth grades engaged in a short- 
term reading intervention for part of each 
day. Students participated in small-group 
tutoring, phonics instruction, and com- 
puter-assisted instruction. Participating 
students were assigned either to the local 
school district or to a private for-profit 
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organization specializing in academic 
interventions for students. Though all of 
the students participating in the program 
showed gains when compared to non- 
participating students, those gains were 
strongest for first graders, who gained the 
equivalent of 6.5 months in reading pro- 
ficiency. These gains may be a reflection 
of a good match between the content and 
the critical elementary school period — an 
important point for program developers to 
consider. 

Though these findings reiterate the impor- 
tance of early intervention for struggling 
readers, students in the upper elementary 
and middle school grades who have diffi- 
culty reading should not be ignored. Most 
research suggests that gains are greatest 
for younger and older students, but a few 
programs have shown gains for upper ele- 
mentary and middle school students. OST 
developers might turn to these particular 
programs for guidance in working with 
students in the middle grades. 

An after-school homework assistance 
program studied by Leslie (1998), for 
example, which was designed to provide 
support and assistance to sixth-, seventh- 
and eighth-grade rural students from 
poor families in Georgia, showed positive 
results. This after-school program incor- 
porated a variety of staff and instructional 
approaches. The activities were led by 
a group of volunteer tutors, including 
classroom teachers, paraprofessionals, and 
middle and high school students. The 
adult tutors communicated with the stu- 
dents’ classroom teachers on a regular 



basis to ensure the most appropriate use 
of homework -help time. Tutors had access 
to computers and software that aided them 
in meeting the needs of the study partici- 
pants. The variety of tutors and the variety 
of methods they used encouraged a level 
of individualization that the participants 
would probably not have experienced in 
their regular classrooms. 

Students attended these tutoring sessions 
for 90 minutes, twice each week. They 
spent the first half-hour on homework as- 
signed by their regular classroom teachers, 
and relied on tutors for assistance. The 
final hour of class was spent reinforcing 
the reading and mathematics objectives 
that had been taught to students during 



Georgia After-School 
Program 



Location: 

• Georgia 

Students served: 

• Sixth-, seventh-, and eighth- 
grade rural students from poor 
families 

Program components: 

• After- school program 

• Volunteer tutors 

• Alignment between school-day 
and after-school work 

Program outcomes: 

• 3 1 percentile point increases 
for sixth- and eighth-graders on 
standardized tests in reading 
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the regular school day. Tutors provided 
assistance to students as they worked on 
skill development sheets and computer' 
based activities designed to strengthen 
their mathematics and grammar skills. 
Some of the curriculum covered within 
the program was provided by classroom 
teachers in the form of lessons and activity 
sheets. Following their year-long participa- 
tion in the program, students showed sig- 
nificant gains on standardized test scores 
in reading: 31 percentile points for both 
sixth and eighth graders, and 49 percentile 
points for seventh graders. This middle 
school program did not fit into the grade 
span during which OST appears to be most 
effective, but it did have other character- 
istics that research has shown to positively 
influence student learning — for instance, 
tutoring and computer-assisted instruction. 

When developing OST programs, program 
staff also should keep in mind the im- 
portance of implementing a well-defined 
reading curriculum, in particular one that 
addresses state standards. One OST study 
that offers guidance is Hausner’s (2000) 
study of the Project Accelerated Literacy 
(PAL) program. PAL includes eight major 
components of literacy instruction, which 
are modeled on scaffolded learning and 
a constructivist approach to reading and 
writing: read aloud to children, shared 
reading, guided reading, independent 
reading, modeled writing, shared writing, 
guided writing, and independent writing. 
Features of the PAL program include (1) 
small class size, (2) a variety of learning 
centers integrating literacy tools and tasks 
(e.g., play office, art center, cooking, and 



Project Accelerated 
Literacy (PAL) 



Location: 

• Not disclosed 

Students served: 

• At-risk kindergarten students 

Program components: 

• Eight components of literacy 
instruction, modeled on 
scaffolded learning and a 
constructivist approach 

• Small class size 

• Variety of learning centers 
(e.g., play office) 

• Two-hour block of time for 
literacy instruction 

• Large- and small-group and 
individual instruction 

• Teaching practices aligned with 
standards 

• Scaffolded teaching 

• Thematic curriculum 

Program outcomes: 

• More than 16 percentile point 
gain in literacy 

book corner); (3) a two-hour block of time 
for literacy instruction using large-group, 
small-group, and individual instruction; 
(4) teaching practices based on each stu- 
dent’s performance on standards; (5) scaf- 
folded teaching that follows a sequence of 
modeling, guiding, observing, and practic- 
ing skills for students; and (6) a thematic 
curriculum (e.g., foods, sea life, and com- 
munity helpers) reflected in each activity 
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center. As a result of this 30'week, half-day 
program, at-risk kindergarten participants 
demonstrated a gain of more than 16 per- 
centile points in literacy. 

On the upper end of the grade spectrum, 
Rembert, Calvert, and Watson (1986) 
evaluated the Summer Camp for Academ- 
ic Development (SCAD), an academic 
summer camp for 10th-, 11th-, and 12th- 
grade students with “evidence of college 
level academic potential, but low motiva- 



Austin Parks & 
Recreation Dept. 
After-School Program 



Location: 

• 20 schools in Austin, TX 

Students served: 

• Low- income children in grades 
3-6 

Program components: 

• Mixed academic & social 
components (e.g., gardening, 
cooking, field trips) 

• Emphasized recreation and 
social time 

• Recreational classes enhanced 
academic skills 

Program outcomes: 

• 12 percentile point achievement 
gains in reading & mathematics 

• Improved student self-esteem 

(Mathematics results for the program 

are discussed in Chapter 3.) 



tion or intention toward postsecondary 
education” (p. 376). For three to four 
weeks for each of two to three summers, 
students lived in dormitories on a college 
campus, attended classes, used college 
library facilities, and experienced a college 
atmosphere. The college preparation 
classes focused on skill mastery in basic 
academics and simulated college instruc- 
tion. Assistance with career planning and 
study skills instruction also was provided. 
Compared to nonparticipants, students in 
this academic summer camp demonstrated 
higher reading achievement (more than 19 
percentile points) and were more likely to 
enter college. It appears that when college 
is only an abstract concept, at-risk youth 
may not be as motivated to achieve aca- 
demically in order to attend college. But 
making the notion of postsecondary edu- 
cation real to these students — and giving 
them the sense that they can be successful 
— may have increased their motivation to 
attend college. 

The freedom and flexibility to address issues 
such as motivation, and to tailor instruc- 
tion to specific student needs, may in fact 
be part of the increasing interest in OST 
programs. Interestingly, some research 
recommends similarly flexible approaches 
to in-school time instruction. For example, 
in a review of the research on reading pro- 
grams in general, Allington (2001) recom- 
mends that conversations about books take 
on the characteristics of out-of-school con- 
versations. In out-of-school conversations 
about reading, participants tend to make 
connections between books, articles, and 
other materials and their own experiences. 
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The value of designing programs and strat- 
egies that engage students in learning is 
that increased time on task can positively 
affect student achievement. 

A Question of Duration 

Although it is natural to assume that longer 
programs produce a larger effect on student 
achievement, this is not always the case. In 
fact, programs shorter than 44 hours and 
longer than 210 hours have been found to 
result in very small negative effects. The 
ideal program length for improving reading 
achievement appears to be between 44 and 
84 hours; McREL’s analysis found that 
children participating in OST programs for 
44 to 84 hours achieved the greatest gains, 
a full 10 percentile points. In programs 
ranging from 85 to 210 hours, the perfor- 
mance increase observed was 7.5 percen- 
tile points (Lauer et al., 2004), less than 
medium-length programs but nonetheless 
a noteworthy performance improvement. 

There are a number of possible reasons for 
the differences in gains. A program that 
lasts fewer than 44 hours might not be long 
enough to fully engage students and influ- 
ence achievement in reading. It may also 
be difficult to sustain the conditions that 
promote student learning over a longer 
period of time, as indicated by the level 
of achievement gains found for programs 
longer than 210 hours. However, although 
programs more than 210 hours long may 
in fact be less effective — either because 
of a loss of focus, attrition of students, or 
for some other reason — studies indicate 
that how often students participate in 
after-school programs can impact student 



achievement. As OST program develop- 
ers established the preferred length of their 
programs, it is important to take these re- 
search findings into account. 

Baker and Witt’s (1996) evaluation of two 
after-school programs in Austin, Texas 
for elementary-school children yielded 
findings that point to the value of high 



Summer Camp for 
Academic 

Development (SCAD) 



Location: 

• South Carolina 

Students served: 

• Tenth-, eleventh-, and twelfth- 
grade students who exhibited 
low motivation toward 
postsecondary education 

Program components: 

• College preparation classes 
conducted on a college campus 

• Mastery in basic academics 
emphasized 

• Assistance with career planning 
and study skills 

Program outcomes: 

• More than 19 percentile- 
point difference in 
reading achievement 
between participants and 
nonparticipants 

• Participants more likely to 
attend college 

(Mathematics results for the 

program are discussed in 

Chapter 3.) 
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levels of participation in OST programs. 
These after-school programs were aimed 
at increasing student interest and engage- 
ment in learning by presenting academi- 
cally oriented activities in the context of a 
goal-oriented, fun, recreational experience. 
Each of the after-school programs offered 
a different balance of activities — ranging 
from primarily academic activities to 
primarily recreational activities — in a 
series of multi-week sessions. Students 
could sign up for one or several activities 
each session or for several activities. Baker 
and Witt found that students who partici- 
pated in three or more activities had higher 
reading grades than students who did not 



Summer Literacy 
Day Camp 



Location: 

• Los Angeles, California 

Students served: 

• Exiting first-grade students from 
predominantly low- income 
schools 

Program components: 

• Mix of academic and 
recreational activities 

• Two hours of reading 
instruction per day, taught by a 
credentialed elementary school 
teacher 

• Tutoring by volunteers for at 
least one hour per week 

Program outcomes: 

• 26 percentile-point gain in 
reading achievement 



participate at all; students who partici- 
pated in five or more activities had higher 
grades than students who participated in 
fewer than five activities. In addition to 
increased participation, other factors, such 
as the quality of student-staff interaction, 
may have contributed to the success of 
these programs. Baker and Witt offer the 
observation that program leaders “tried to 
use the after-school activities to provide 
quality contact time and highlight the 
linkage between fun and learning” (p. 18). 
The success of this multi-session program 
suggests that variety should be a consider- 
ation in OST program design. 

Choosing OST Strategies 

In a review of extended-day and after 
school programs, Fashola (1998) recom- 
mends that OST program developers 
“adopt or create well-structured programs 
that provide extensive training” (p. 53). 
As part of this process, program developers 
must consider which research-based strate- 
gies are best suited to their students’ needs. 
One-on-one tutoring is one such strategy 
to consider. 

Research shows that working with students 
one-on-one results in the largest achieve- 
ment gains, 19 percentile points (Lauer et 
al., 2004). Schacter’s (2001) evaluation 
of the impact of an eight-week, summer 
day camp that implemented a systematic 
reading curriculum illustrates the potential 
for gains from one-on-one tutoring. The 
purpose of the camp was to increase reading 
achievement for disadvantaged first graders. 
Children received two hours of reading in- 
struction each day; the remainder of each 
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day was spent in recreational activities 
such as swimming, arts and crafts, music 
and dance. Participants showed significant 
gains in reading improvement — a more 
than 26 percentile-point increase. 

In Morris, Shaw, and Perney’s (1990) study 
of an after-school tutoring program, 60 low- 
achieving second and third graders were 
randomly assigned to either after-school 
tutoring or a control condition of no tutor- 
ing. The after-school tutoring was provided 
for one year by community volunteers who 
were supervised by reading specialists. The 
supervisors designed each tutoring lesson, 
and the tutors implemented the lessons 
and recorded their observations for the 
supervisor who, in turn, designed subse- 
quent lessons. Tutorial strategies included 
shared reading, word study, reading books, 
and writing stories. Attaining the overall 
positive effect of after-school tutoring on 



reading achievement (a 19 percentile- 
point gain) required 50 hours of “well- 
planned, closely supervised one-to-one 
tutoring” (p. 147). 

Notably, each of these strategies has 
at its core a focus on differentiat- 
ing instruction. One benefit of OST 
programs may be that they provide 
the time and flexibility needed to 
accomodate students’ different learning 
styles and academic and developmental 
needs. 



The Howard Street 
Tutoring Program 



Location: 

• Chicago, IL 

Students served: 

• Low-achieving second and 
third graders 

Program components: 

• Year-long, after-school tutoring 
program 

• Community volunteers served 
as tutors, supervised by reading 
specialists 

• Tutorial strategies included 
shared reading, word study, 
reading books, and writing 
stories 

Program outcomes: 

• 19 percentile point gain in 
reading achievement 
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Sustaining Gains Over Time 

Some studies aimed at identifying strategies 
for closing the achievement gap indicate 
that OST strategies can be effective at pre- 
venting learning loss among low-achieving 
students, especially during the summer 
months. In fact, Borman, Rachuba, Fairch- 
ild, and Kaplan (2002) reported a cumula- 
tive impact on the learning of kindergarten 
and first-grade students in summer school 
over a period of two and three years. 

In contrast, other studies have found that 
students did not sustain learning gains over 



Key findings 



Research indicates student gains 
are greatest when reading-based 
OST programs 

0 Offer one-on-one tutoring or 
mixed student groupings 

0 Range from 44-84 hours 

0 Focus on early elementary & 
high school students 



time. For example, Hausner’s (2000) evalu- 
ation of the PAL after-school kindergarten 
literacy program reported that low-per- 
forming students’ literacy scores increased 
significantly (by more than 16 percentile 
points) but that these students did not 
show sustained improvement in the second 
grade. The author suggested that at-risk 
students may need more than one literacy 
intervention to retain the gains made as a 
result of the early intervention program. 

Implications for Program 
Developers 

McREL’s analysis of the available high- 
quality research reveals an overall ten- 
dency for improved reading achievement 
among low-achieving or at-risk students 
who participate in OST programs. This 
suggests that policymakers and practitio- 
ners should consider OST programs as a 
promising option for boosting the reading 
achievement of these student groups. 
Based on the body of research on OST 
strategies in reading, some conclusions can 
be made related to program design and ef- 
fective practice. 

Most notably, the following OST strategies 
and characteristics appear to be effective 
in improving the reading achievement of 
low-achieving students: 

• Tutoring and mixed student groupings 

• Programs between 44 and 84 hours in 
length 

• Programs that focus on early elemen- 
tary and high school students 
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As program developers consider how best 
to meet the needs of their at-risk students, 
these findings should be kept in mind. 
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CHAPTER E 



Effective OST 
Mathematics Programs 



P roficiency in mathematics is a 
critical life skill. Mathematics 
knowledge, as well as essential 
problem-solving and reasoning 
skills, are essential for success in many 
business and technical careers, and support 
a host of other endeavors as well. In many 
cases, however, our nation’s students are 
not learning these skills: just 32 percent 
of fourth graders and 29 percent of eighth 
graders performed at or above the profi- 
cient level in mathematics on the 2003 
National Assessment of Educational 
Progress (NAEP). The picture is more 
dismal for at-risk students: at both grade 
levels, students from low-income families 
scored significantly lower in mathemat- 
ics than their more wealthy counterparts 
(National Center for Education Statistics, 

2004). 



What kinds of mathematics knowledge 
and skill are required of students? NAEP 
assessments evaluate students’ conceptual 
understanding, procedural knowledge, and 
problem-solving skills within five broad 
strands of mathematics content: number 
sense, properties, and operations; measure- 
ment; geometry and spatial sense; data 
analysis, statistics, and probability; and 
algebra and functions (National Center for 
Education Statistics, n.d.). 

The National Council of Teachers of 
Mathematics (NCTM) has created a set 
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of-School Time Mathematics 
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of standards that outline the content and 
processes that children should know and be 
able to do across grade bands. For instance, 
the “number and operations” standard states 
that children should understand numbers, 
ways of representing numbers, relationships 
among numbers, and number systems. In 
the prekindergarten through the second 
grade band, for example, this means that 
children should count with understanding 
and understand and represent commonly 
used fractions, among other skills (NCTM, 
n.d.). 

In addition, the National Research Council 
identifies five interdependent and necessary 
“strands” of mathematical proficiency: 

• Understanding: Comprehending math- 
ematical concepts, operations, and 
relations — knowing what mathemati- 
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cal symbols, diagrams, and procedures 
mean. 

• Computing: Carrying out mathemati- 
cal procedures, such as adding, sub- 
tracting, multiplying, and dividing 
numbers flexibly, accurately, efficiently, 
and appropriately. 

• Applying: Being able to formulate 
problems mathematically and to 
devise strategies for solving them 
using concepts and procedures appro- 
priately. 

• Reasoning: Using logic to explain 
and justify a solution to a problem or 
to extend from something known to 
something not yet known. 

• Engaging: Seeing mathematics as sen- 
sible, usable, and doable — if you work 
at it — and being willing to do the 
work (Kilpatrick & Swafford, 2002, p. 
9). 

The NAEP and National Research Council 
frameworks, and the NCTM standards, 
all illustrate the importance of bringing 
students to deep levels of learning and 
understanding in mathematics. To help 
students achieve this degree of proficiency 
and understanding, many schools are 
exploring new mathematics curricula and 
providing teachers with the professional 
development necessary to enhance their 
ability to teach students with a broad range 
of needs. For at-risk students, however, 
these in-school efforts may not be enough. 
Many of these students need more time and 



personalized attention than that available 
during regular school hours. Many schools 
and districts are turning to out-of-school 
time (OST) programs for solutions. 

Though the research on OST mathematics 
programs is still emerging, there are 
a number of key Endings for program 
developers to consider. This chapter 
presents those Endings, along with other 
guidance and information about enhancing 
the mathematics achievement of at-risk 
students. 

Overview of Research on Out-of- 
School Time Mathematics Programs 

Do OST programs make a difference in 
mathematics achievement — and if so, how 
do they make a difference? The available 
research offers guidance for educators on 
both counts. McREL’s review of 33 OST 
program studies found that, on average, 
at-risk students who participated in OST 
programs scored more than six percentile 
points higher on mathematics achievement 
tests than students who did not participate 
(Lauer et ak, 2004). 

At Erst glance, six percentile points might 
seem to be a small increase, but all OST 
programs are not created equal. Some 
program characteristics seem to lead to 
greater impacts on student achievement 
than others. Specifically, McREL found 
that the following program structures and 
characteristics resulted in the largest gains 
in student achievement: 

• Programs that combine mathematics 
instruction and social activities 
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• Programs that last between 45 and 100 
hours 

• Programs for middle school students 

• Programs for high school students 

These results offer some interesting points 
for program developers to consider. By more 
closely exploring the programs that resulted 
in larger effects on student achievement, 
schools may be able to maximize the effects 
of their OST programs. 

The Value of Social Activities 

Programs outside the typical school day 
that focus solely on academics are not the 
only way, or perhaps even the most effective 
way, to improve students’ mathematics 
achievement. A number of studies point to 
the value of weaving social activities and 
life-skills instruction into OST programs. 
For example, Branch’s study (1986) of the 
Summer Education and Training Program 
(STEP) reported strong mathematics gains 
among participating students. Between 1984 
and 1991, STEP served 20,000 adolescents 
in five urban locations across the nation. 
The program was added to an existing 
federally funded summer-jobs program, the 
Summer Youth Employment and Training 
Program (SYETP), which served disadvan- 
taged 14- and 15 -year-olds. Branch notes 
that SYETP “had only limited success 
in helping [participants] make successful 
transitions into the world of work” (p. ii), 
so STEP was created to help bridge the gap. 
Program components included academic 
remediation, life-skills instruction, summer 
employment, and support during the 
regular school year. 



STEP Program 



Location: 

• Five urban locations across the 
nation 

Students served: 

• Disadvantaged 14-15 year olds 

Program components: 

• Academic remediation 

• Life-skills instruction 

• Summer employment 

• Support during regular school 
year 

Program outcomes: 

• Initial participants outscored 
peers by 1/4 grade equivalent in 
reading & mathematics 

• Academic effects relatively short 
lived 

STEP’s 15-month timeframe spanned 
two summers and the interim school year. 
Participants divided their summers equally 
between working and learning. Each 
student held a part-time job for approxi- 
mately 90 hours, and spent the same amount 
of time in remediation classes. Participants 
also took part in a life-skills class that met 
two mornings each week. Over the summer, 
participants received highly individualized, 
computer-assisted instruction in reading 
and mathematics. Program teachers kept 
track of student progress through a variety 
of diagnostic tests that were closely aligned 
to program goals. The life-skills program 
component addressed students’ social 
needs through a wide range of instructional 
formats — including speakers, discussion, 
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and creative activities. These classroom 
components were designed to complement 
the existing summer jobs program that 
provided participants with office, parks 
and recreation, or custodial minimum-wage 
work. 



Researchers suggested the 
STEP program was ineffective at 
providing a school-year component 
to support lasting academic gains. 



The first group of participants in STEP 
made impressive gains, outscoring their 
peers by approximately one-quarter of 
a grade equivalent in both reading and 
mathematics. A later 
study of the program, 
however, found the 
academic effects to be 
relatively short lived 
(Grossman & Sipe, 

1992). The researchers 
suggested that the 
STEP program had been 
ineffective at providing 
the planned school-year 
component to support 
lasting academic gains 
— perhaps an important 
cautionary note for 
program developers to 
consider. If, in fact, this 
program did not result 
in sustained achievement gains because 
students did not receive additional help 
during the school year, this emphasizes the 
need to provide ongoing help to students at 
risk of failure. 



In another program, children participating 
in after-school sessions administered hy the 
Austin Parks and Recreation Department 
and the Austin Independent School District 
(Baker & Witt, 1996) realized achievement 
gains of 12 percentile points in reading 
and mathematics. Outcomes other than 
academics were measured as well; student 
gains were evident not only in reading 
and mathematics, but also in students’ self- 
esteem. This after-school program served 
third- through sixth-grade children from 
20 schools in low-income communities. In 
addition to household income levels, the 
communities featured in Baker and Witt’s 
(1996) evaluation exhibited other risk 
factors. These included lower levels of per 
capita income, higher 
rates of unemployment, 
higher percentages of 
families with children 
under 18 living below 
the poverty level, higher 
rates of Spanish spoken 
in the home, and lower 
rates of high school 
graduation. 

Classes and activities 
in the Austin program 
were taught by teachers 
who received additional 
pay to take on the task, 
as well as instructors 
hired by the Austin 
Parks and Recreation Department. The 
classes were free to participants; children 
were accepted on a first-come, first-served 
basis. The program mixed academic and 
social components in a variety of classes. 
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In a class with an agricultural focus, for 
example, students learned about gardening 
and cooking in hands-on sessions that 
touched on academic areas such as the 
earth sciences and measurement. Another 
class, the Explorers, provided opportunities 
for students to learn about and see firsthand 
a variety of animal and plant life that are 
not typically encountered in the Austin 
area. Class activities included field trips to 
local zoos, parks, and to the seashore. 

The Austin program also offered classes 
that emphasized recreation and social time, 
leveraging, for example, students’ interest 
in sports, Hispanic culture, and creative 
activities, such as learning to be a clown. 
Even these recreational classes had an eye 
toward academics, however. For example, 
a magic activity was designed to enhance 
mathematics skills, and sports classes 
included the calculation of sports statistics. 

Though adding social components to 
OST mathematics programs appears to be 
effective, the underlying academic content 
must be paired with effective instructional 
strategies. For example, is the program 
comprehensive and aligned with state- and 
national-level standards? Does the program 
focus on conceptual understanding, as well 
as procedural fluency? Does the program 
include well-developed ideas that build on 
other concepts within and across grades? 
Do program instructors have a deep under- 
standing of the subject matter (Apthorp et 
al., 2001)? Though these considerations are 
not specific to OST programs, it is critical 
that program developers examine a wide 
range of research before making decisions 



about program structures and content. 

Programs That Are Just the Right 
Length 

As was the case with the reading research, 
McREL found in its review of OST 
mathematics programs that duration is 
one indicator of program effectiveness. 
Programs providing fewer than 45 hours of 
total intervention time appear to result in 
less of an impact on student achievement 
than those ranging from 45 to 100 hours 
(Lauer et al., 2004). 

Programs ranging from 45 to 7 00 

hours appear to be most effective. 

One such program is described by Kociemba 
(1995), who examined the impact of the 
1988 Minneapolis Public School Summer 
Program on student achievement in 
mathematics and reading for children in 
second and fifth grades. The students par- 
ticipating in the program had either failed 
the Minneapolis Public School Benchmark 
Test in reading and mathematics, or were 
performing well below grade level and were 
recommended to the program by a teacher 
or counselor. Students attended 24 days 
of summer school, for four hours per day. 
Instruction focused on reading comprehen- 
sion and mathematics computation skills, 
and included both small group and indi- 
vidualized instruction. 

Following their participation in the program, 
both second and fifth grade students showed 
gains in mathematics achievement on a 
standardized test. Second-graders made 
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Minneapolis Public 
School Summer Program 



Location: 

• Minneapolis, MN 

Students served: 

• Grade 2 & 5 children failing 
state test, performing below 
grade level, or recommended 
by teachers or counselors 

Program components: 

• 24 days of summer school, 4 
hours/day 

• Small group & individualized 
instruction focused on math 
computation skills 

Program outcomes: 

• 2nd graders made 3 percentile 
point gains 

• 5 th graders made 15 percentile 
point gains 



gains of approximately 3 percentile points; 
fifth graders made gains of 15 percentile 
points. 

As educators develop new OST programs, 
there are a number of things to keep 
in mind in terms of program duration. 
Although programs ranging from 45 to 100 
hours appear to be most effective, it is up to 
the program developers to determine which 
program duration is most appropriate for 
their schools. Program developers might 
prefer, for instance, to develop a series 
of shorter interventions rather than one 
longer, sustained program. There is some 
research to support the utility of this 



approach. The program evaluated by Baker 
and Witt (1996), for example, describes 
schools that divided their interventions 
into three or four 6- to 9-week sessions, 
resulting in significant achievement gains. 
Though shorter programs may require more 
logistical support — designing different 
approaches, assigning students to classes, 
identifying and training staff — they also 
have the potential to be more exciting and 
engaging for students. 

Effective Middle and High School 
Programs 

The available research on mathematics 
programs for at-risk students uncovered 
unexpectedly large effects in middle and 
high schools. Middle school programs, on 
average, produced a 6 percentile -point gain 
in achievement, and high school programs 
produced an average 17 percentile-point 
gain. Although research does point to 
effective elementary school mathematics 
programs, more examples are available of 
effective OST programs for students at 
middle school and high school levels. This 
prevalence of math-focused OST programs 
for older students may simply be a function 
of the relative complexity of mathematics 
(e.g., pre-algebra, algebra) in the middle and 
high school curricula. Also, this emphasis 
may reflect the structure of middle and high 



Middle school programs, on 
average, produced a 6 percentile- 
point gain in achievement, and 
high school programs produced 
an average 17 percentile-point 
gain. 
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schools, where students must repeat failed 
courses before moving on to the next level 
of mathematics. Middle and high school 
students are also under increased pressure 
to prepare for college. The focus of the 
research to date may be a reflection of these 
students’ needs. 

One example of a program for older 
students is the Summer Camp for 
Academic Development (SCAD), in 
which a population of at-risk students was 
identified and participated in a summer 
academic program (Rembert, Calvert, & 
Watson, 1986). This program was a grant- 
supported program on the Winthrop 
College campus, located in Rock Hill, 
South Carolina. Participants were a group 
of African-American high school students 
who had the academic potential to succeed 
in college, but who were unlikely to enroll 
— either because of a lack of motivation or 
because of socioeconomic factors. Students 
from central South Carolina traveled to 
Winthrop College for four summers in the 
1980s, and attended several weeks of college 
classes. 

Students participating in SCAD engaged 
in study throughout the school day on 
the Winthrop campus. From Monday 
through Thursday, their schedules were 
filled with classes encouraging interest and 
mastery in core subjects including reading, 
writing, mathematics, and science. The 
classes, which were taught by public school 
teachers and graduate students, incorpo- 
rated approaches such as computer-assisted 
instruction, science laboratory work, and 
library research in an effort to acclimate 



Summer Camp for 
Academic Development 
(SCAD) 



Location: 

• Winthrop College, Rock Hill, 
SC 

Students served: 

• High school students who 
exhibited the potential for 
college success 

Program components: 

• Summer classes on college 
campus 

• Encouraged interest & mastery 
in core subjects (reading, 
writing, math, science) 

• Incorporated computer-aided 
instruction, science labs, library 
research, field trips 

Program outcomes: 

• 13 percentile point gain in 
mathematics achievement 

• 69% of participants enrolled in 
college 



the participants to college coursework. 

Fridays and Saturdays were reserved for 
seminars and field trips. Seminars were 
led by Winthrop counselors and focused 
on career opportunities, study skills, and 
self-esteem. These issues were reinforced 
by adult mentors assigned to participants, 
who served as role models for professional 
success and community leadership. These 
African American adults were not only able 
to speak to participants about the career 
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paths — business, finance, law, medicine 
— in which they had achieved success, 
but they were also able to address the 
challenges they faced along the way. Most 
of participants’ field trips exposed them 
to local industries and businesses, thereby 
enriching the career focus of the program. 

The college experience was a complete 
one for these students. Beyond the classes 
and class activities, while on campus the 
participants had access to a variety of 
college services, including the sports and 
recreation facilities, the student center, 
computer labs, and even the guidance of 
college professors and administrators. The 
participants also got a taste of dorm life, 
calling the Winthrop dorms “home” during 
their three or four weeks on campus. 

Rembert et al.’s evaluation of the SCAD 
program reported a 13 percentile -point gain 
in students’ mathematics achievement. In 
addition to enhancing African American 
students’ skills and fostering their interest 
in academics, the SCAD program also was 
successful in its primary goal: to encourage 
students to go to college. In the years that 
followed, 69 percent of program graduates 
enrolled in college, a quarter of them in 
Winthrop. Many program graduates also 
went on to earn degrees and graduate 
degrees (see sidebar). 

The SCAD program is not the only 
program of its type. Another example 
of a summer college program for urban 
high school students is the Twenty-first 
Century Mathematics Center for Urban 
High Schools at Temple University. The 



program offered algebra, geometry, and 
functions classes that served at-risk 
students. Participants were students from a 
public magnet school who enrolled in math 



FROM THE FIELD: THE 
SUMMER CAMP FOR 
ACADEMIC DEVELOPMENT 
PROGRAM 

" Several of the participants of 
SCAD went on to pursue graduate 
studies here at Winthrop and/or 
elsewhere and are doing quite 
well, i cannot tell you how 
proud I am of their individual 
and collective successes. Almost 
all of them were first generation 
college students who, when given 
the opportunities to demonstrate 
their academic potential, readily 
accepted and demonstrated 
their gifts and talents. Who in the 
mid-80s would have predicted 
that these students from rural 
South Carolina with limited 
access to higher education would 
be bankers, counselors, and 
sophisticated investors today ? 
Perhaps, many in their respective 
families or communities knew 
they had potential, but did not 
realize the absolute jewels they 
were and are. They shine so very 
brightly today. " 

Wilhe/menia i. Rembert, Ph.D. 
Associate Vice President 
Graduate Studies, Winthrop 
University 
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classes designed for the program and offered 
by Temple. 

Evaluations of the program indicated that 
participants showed significant gains in 
mathematics achievement — a 34 percentile- 
point gain for female participants and a 
30 percentile-point gain for males (Riley, 



Twenty-first Century 
Mathematics Center for 
Urban High Schools 



Location: 

• Temple University, Philadelphia, 
PA 

Students served: 

• At-risk urban high school 
students 

Program components: 

• Summer college program 

• Math (algebra, geometry, & 
functions) classes taught by 
university faculty 

• In-class lectures, take-home 
worksheets 

• Graduate students & program 
graduates served as tutors 

• Focused on both academies and 
building student confidence in 
ability to succeed 

Program outcomes: 

• 34 percentile-point gains for 
females 

• 30 percentile-point gains for 
males 



1997). The Temple program differed from 
the SCAD program in a number of ways. 
Participants in the Temple program listened 
to a topic lecture and then completed 
both in-class and take-home worksheets. 
Lectures were followed by in-class tasks and 
homework assignments that were supported 
by available graduate student and program 
graduate tutors. To help participants meet 
the expectations for the amount of content 
and the level of mastery expected, graduate 
students and program graduates provided 
small-group and individual tutoring before, 
during, and after classes. 

In addition to grade level, the SCAD and 
Temple programs share another distinguish- 
ing characteristic worth noting. In each 
case, the program designers were focused 
on meeting the needs of at-risk students 
who not only may have had trouble with 
complex concepts, but who also struggled 
with motivational issues. These two 
programs emphasized teaching core subjects 
and building students’ confidence in their 
ability to succeed. Though the approaches 
taken by these two programs were different, 
the results were quite similar. In each case, 
the large effect sizes appear to indicate that 
the programs were successful in helping 
students recognize that they were able to 
master complex material under pressure. 
In the SCAD program this pressure was a 
result of the heavy class load, whereas in 
the Temple program this pressure was the 
product of the pace of the courses. But, in 
both cases, the students appear to have 
left the programs with the confidence of 
knowing that they can succeed. 
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Implications for 
Designers of OST 
Mathematics Programs 

McREL’s review of the 
research yields a number 
of considerations for OST 
mathematics program 

developers to keep in mind. 

The research indicates that 
gains for OST program 
participants were greatest 
for programs that combine 
mathematics and social 
emphases, programs that 
range between 45 and 100 
hours, and programs that target middle and 
high school students. These findings offer a 
starting point for developers of new OST 
mathematics programs. 

As staff go about developing OST programs 
in mathematics, it is important to ensure 
that appropriate research-based strategies 
are in place to maximize their impact 
and the potential for long-term student 
performance gains. The Kociemba (1995) 



Key findings 



Research indicates student gains 
are greatest when math-based OST 
programs ... 

0 Combine math & social emphases 

0 Range from 45-100 hours 

0 Target middle & high school 
students 



study described in this chapter offers 
an interesting illustration of a successful 
OST program. This program addressed 
what appears to be a key element of OST 
program success — duration. This program 
also incorporated tutoring and aligned 
the after-school program to the school-day 
curriculum, which may well have had an 
additional impact on its success. 

The programs examined in this chapter 
have important characteristics in common: 
each had high expectations for participants, 
and provided the instruction and supports 
necessary for students to be successful. If 
OST program developers provide these 
supports, incorporate research-based 
strategies for instruction, and keep in 
mind the other factors that have been 
shown to impact participant achievement, 
their programs have a greater potential to 
positively influence student achievement. 
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Moving Forward: 
Developing and 

r o 

Administering 
OST Programs 



T hough out'of'schoohtime programs 
have been a part of our nation’s 
history since as early as the 1800s, 
for a long time these programs were 
viewed as merely providing childcare or 
social activities. Only recently have edu- 
cators begun to look to them as a means for 
helping all students meet high academic 
standards. Perhaps in part because of this 
recent shift in focus, the research on these 
programs is not yet definitive. Nonetheless, 
McREL’s review of the available research 
reveals a few key points for administrators 
involved in the design and implementa- 
tion of programs beyond the traditional 
school day: 

The timeframes for delivering OST programs 
(e.g., after school, summer school) do not 
appear to influence the effectiveness of OST 
strategies. Policymakers and educators 
should look at other factors — for example, 
program duration and cost, as well as staff 
recruitment, program location, and other 
implementation issues — when making 
decisions about the delivery of programs 
outside the traditional school day. What 
matters is not when the programs are 
implemented, but how they are imple- 
mented. 

Students in early elementary grades and 
in high school are more likely than older 



elementary and middle school students to 
benefit from OST programs to improve 
reading. However, middle and high school 
students are most likely to benefit from OST 
programs in mathematics. The findings for 
reading achievement in part support prior 
research on the importance of early inter- 
vention and reading skill development; the 
results for mathematics suggest that stu- 
dents who struggle with complex subject 
matter can make significant achievement 
gains through the extra help provided in 
OST programs. 

OST programs need not focus solely on 
academic activities to have positive effects on 
student achievement. Study results indicate 
that OST programs in which activities are 
both academic and social can positively in- 
fluence student achievement. This finding 
supports the belief that OST programs 
should address the developmental needs 
of the whole child (Halpern, 2002) and 
offer a variety of activities (Miller, 2003). 
However, the results also suggest that ef- 
fectiveness related to program focus may 
vary depending on grade level and content 
area. 

Administrators should monitor student learn- 
ing and the implementation of OST programs 
to determine the appropriate investment of 
time for specific strategies and activities. Al- 
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though McREL’s synthesis indicates that 
OST programs need to be at least 45 hours 
long to be effective, longer OST programs 
do not necessarily lead to additional gains. 
The optimal length of a program may 
depend on the content area and other 
factors. This result supports other findings 
that extending the time for learning does 
not mean that students will be engaged 
in learning during that additional time 
(WestEd, 2002). 

OST reading strategies that provide one- 
on-one tutoring for at-risk students can 
lead to particularly strong results in terms 
of student achievement. This was one of 
the strongest findings from McREL’s meta- 
analysis and is supported hy other research 
on tutoring of at-risk students during the 
school day (Barley et al., 2002; Elbaum et 
al., 2000). OST programs that have reading 
improvement as a goal should provide indi- 
vidual tutoring of students. 

These findings are not intended to serve 
as the last word on OST programs, but do 
offer educators points to consider as they 
work on the design and implementation 
of these programs. Part of the appeal of 
OST programs may well be their flexibility. 
The importance of good program design is 
paramount. When working with students 
outside the traditional school day, educa- 
tors have more leeway to incorporate one- 
on-one tutoring and other strategies that 
positively impact student achievement. 
OST programs can help us, as educators, 
move closer to our primary goal: focusing 
our collective attention and resources on 
ensuring that every child graduates from 



high school prepared for success in life, 
work, and future education. The recom- 
mendations in this issue of Noteworthy are 
intended to provide guidance for educators 
as they work toward that goal. 
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Chapter 4: Moving Forward: Developing and Administering OST Programs 



Checklist for Program Developers: Things to Consider 

What resources (time, space, funding, staff) are available that could be used in 
providing OST programming? 



Are there existing resources (technology, district funding, local businesses 
or attractions, people with unique talents, unscheduled facilities) that can be 
tapped? 



Who are the students you would like to participate in your OST program? 



Are students' needs consistent with program goals? 



Will your program need to diagnose individual needs and prescribe appropriate 
instruction? 



What methods will you use to engage, interest, and motivate your at-risk 
student participants? 



Consider not only what approaches might be most effective for specific age 
groups, but also what might be most appealing to different age groups. Given 
the competing demands for their time and attention, programs must be 
engaging to student participants. 



What would you like these students to learn as a result of their participation? 
Clearly articulate your learning goals, and align them with the goals of the 
school. 



In what ways can your OST program enhance the participants' in-school 
experience? 
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